Département d ’ Informatique et de recherche opérationnelle ( DIRO ) Rapport technique 1273 Février 2006 Geo - consistency for Multi - Camera Stereo by Marc - Antoine Drouin

This paper introduces the concept of geo-consistency to overcome the occlusion problems associated with wide-baseline multiple-camera stereo. By expressing more formally the geometric relationship between occlusion and visibility, it provides a new tool to understand and compare the behavior of stereo algorithms. In this context, we present a new visibility mask handling that can render most regular stereo algorithms ”occlusion aware”. Rather than explicitly modeling occlusions in the matching cost function, it detects occlusions in the depth map obtained from regular efcient stereo matching algorithms. The algorithm gradually modies the matching cost function according to the history of inconsistencies in the depth map, until convergence. The nal depth map is guaranteed to preserve the coherence between camera visibility and geometry. We also provide a fast specialized stereo algorithm, based on Iterative Dynamic Programming, that model visibility very efciently. It is based on partial geo-consistency, in which some of the visibility information is always known exactly. Geo-consistency makes it easier to express the ordering constraint. This is helpful for the detection of the parts of a scene not fullling it. These zones are the most subject to error and should be handled separately. We observed that our fast IDP algorithm is especially wellsuited for high discontinuity areas. For experiments, we applied our general occlusion algorithm to two common graph-theoretic stereo algorithms. The validity of our framework is demonstrated using real imagery taken with various baselines, as it is known that occlusion increases with it.